Pixel interpolating device capable of preventing noise generation

ABSTRACT

A pixel interpolating device using the IM-GPDCT method and capable of preventing noise generation divides an original image into a plurality of blocks and picks up a block to be processed as a target block. Further, an image peripheral to the target block is extracted as a peripheral image. By using the peripheral image as an extension region, DCT transform is carried out. Thereafter, the peripheral image is magnified, and the IM-GPDCT processing is carried by using the magnified image.

This application is based on applications Nos. 10-012532 and 10-029691filed in Japan, the contents of which is hereby incorporated byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to pixel interpolating devices and moreparticularly to a pixel interpolating device using the IM-GPDCT methodfor interpolating a pixel while restoring a high frequency component ofan image.

2. Description of the Related Art

In converting a pixel density (interpolating a pixel) of an image basedon image information included in a sampled original image, a method(“IM-GPDCT method”) is conventionally known which restores a spatialhigh frequency component, which is lost during a sampling process, underthe two restrictive conditions that information on a passing frequencyband is correct and an expanse of an image is limited in a process inwhich normal transformation and inverse transformation of the image arerepeated by orthogonal transformation.

The principle of the method will be described in the following. Anoperation is known which restores an original signal that is lostbecause a frequency band is limited when an original image is sampled.Such an operation is generally accompanied by the super-resolutionproblem.

In any observation system that can physically be implemented, a highfrequency component of at least a certain frequency cannot be observed.

For example, an image pick-up system has a limited size of an entranceaperture, and the image pick-up system itself functions as a low passfilter (LPF). Thus, a large number of frequency components that can bepropagated are lost, and resolution is lowered.

The lost resolution can be obtained only by bandwidth extrapolation(super-resolution problem) in which an original signal prior to passagethrough the image pick-up system is found from an image signal that canbe obtained through the image pick-up system.

The super-resolution problem is mathematically formulated for a functionof one variable as described below. When an original signal in a realspace region is f(x), a signal that is formed by limiting the frequencycomponent band of original signal f(x) to cut-off frequency u0 at mostand that actually passes through an image pick-up system is g(x), andthe process for carrying out band limitation is expressed as A, theexpression (1) below is derived.

g(x)=Af(x)  (1)

The process A corresponds to actual application of an LPF by passing theoriginal signal through the image pick-up system.

The functions that correspond to Fourier transform of signals f(x) andg(x) above are assumed to be F(u) and G(u), and a window function W(u)in a frequency region is defined by the following expressions (2) and(3).

W(u)=1(|u|≦u 0)  (2)

W(u)=0(|u|>u 0)  (3)

Performance of window function W(u) corresponds to application of anideal LPF.

Further, the expression (1) above is expressed in a frequency region asthe expression (4) below.

G(u)=W(u)F(u)  (4)

The super-resolution problem is intended to find original signal f(x)from band-limited signal g(x) by the expression (1) in a real spaceregion and to find F(u) from G(u) of the expression (4) in a frequencyregion.

If original signal f(x) is not limited at all, however, F(u) cannot befound.

Accordingly, the super-resolution problem can be solved by applying aprocess in which unlimited resolution can be obtained in principle whenoriginal signal f(x) is subjected to spatial region limitation so thatan object has a limited size, and f(x) only exists in a certain region,a region between −x0 and +x0, for example, and it does not exist outsidethe region.

Conventionally, the Gerchberg-Papoulis iteration method (GP iterationmethod) is used to solve the super-resolution problem.

FIG. 14 illustrates the principle of the GP iteration method. In FIG.14(A), (C), (E) and (G) correspond to a frequency region while (B), (D),(F) and (H) correspond to a real space region. FIG. 14(B) shows originalsignal f(x) of which region is limited to a space |x|≦x0. FIG. 14(A)shows Fourier transform F(u) of original signal f(x), and F(u) includeseven an unlimitedly high frequency component because the region oforiginal signal f(x) is limited.

FIG. 14(C) indicates that only G(u), which is the part of the space|u|≦u0 of F(u), is observed. In other words, the expression (4) using awindow function such as the expressions (2) and (3) above is formed.

Inverse Fourier transform of G(u) is g(x) in FIG. 14(D). Solving thesuper-resolution problem is to find F(u) or f(x) from G(u) above.

The operation in the GP iteration method will be described in thefollowing. Since the band of G(u) is limited to |u|≦u0, g(x) extendsunlimitedly.

Since it is known that the region of original signal f(x) is limited tothe interval |x|≦x0, however, the same region limitation is performedeven on g(x).

In short, only the part of interval |x|≦x0 in g(x) is extracted toobtain f1(x). When f1(x) is expressed as an expression that uses awindow function w(x) expressed by the following expressions (5) and (6),the expression (7) is obtained. This is function f1(x) shown in FIG.14(F).

w(x)=1(|x|≦x 0)  (5)

w(x)=0(|x|>x 0)  (6)

f 1(x)=w(x)g(x)  (7)

Fourier transform of f1(x) results in F1(u) in FIG. 14(E). Since theregion of f1(x) is limited, F1(u) extends unlimitedly. However, acorrect value of G(u)=F(u) is already known for space |u|≦u0, andtherefore the portion of |u|=≦u0 in F1(u) is substituted by G(u).

The waveform formed in this manner is G1(u) in FIG. 14(G). The relationsare expressed by the expressions (8) to (10). Inverse Fourier transformof G1(u) above is g1(x) in FIG. 11(H).

G 1(u)=G(u)+(1−W(u))F 1(u)  (8)

G 1(u)=G(u)(|u|≦u 0)  (9)

G 1(u)=F 1(u)(|u|>u 0)  (10)

The processing from (C), (D) to (G), (H) in FIG. 14 is the first roundof the GP iteration method. Then, the operation of extracting only theportion of interval |x|=≦u0 from g1(x) in FIG. 14(H), carrying outFourier transform on f2(x) (not shown) corresponding to f1(x) in FIG.14(F), and finding F2(u) (not shown) corresponding to FIG. 14(E) isrepeatedly performed. Thus, an original signal can perfectly berestored.

Conventionally, an operation load is reduced by substituting Fouriertransform in the GP iteration method by discrete cosine transform (DCT).This is called the “IM-GPDCT” method.

FIG. 15 is a flow chart schematically showing a processing flow carriedout in image magnification processing (an example of pixel interpolationprocessing) using the conventional IM-GPDCT method, and FIG. 16illustrates the processing of the flow chart in FIG. 15.

It is assumed here that an original image consisting of N×N pixels shownin FIG. 16(A) is magnified m times to produce an image of (N×m)² pixels.In FIG. 16, the numbers in parenthesis correspond to the step numbers ofthe flow chart in FIG. 15.

Referring to FIG. 15, the number of iteration times in the GP iterationmethod and the value of a magnification rate (resolution conversionrate) are set in step S1. In step S2, an original image to be magnified,shown in FIG. 16(A), is read. In step S3, an image of interest (herein,an image shown in FIG. 16(A)) is extracted.

In step S4, an image extending around the image of interest of N×Npixels (extension region) is found to limit the spatial expanse of theimage. Conventionally, the data of an image to be extended is fixed to aparticular value, and calculation of extension region data is notcarried out. That is, in step S4, predetermined image data “L” is addedas an extension region to the original image, and expansion to an imageof nN×nN pixels shown in FIG. 16(B) is performed. Here, n is a realnumber larger than 1, and n is set so that nmN is a power of 2.

In step S5, the image in FIG. 16(B) is transformed to a frequencycomponent a shown in FIG. 16(C) by two-dimensional DCT transform. Thefrequency component a is known information in the DCT region andcorresponds to a spatial low frequency component.

In step S6, the value of a is stored. In step S7, the frequency band offrequency component a is extended to a high frequency band according toa magnification rate, as shown in FIG. 16(D).

At this time, the high frequency band for expansion is set to an initialvalue 0. The extended frequency region is set to have nmN×nmN pixels.

In step S8, inverse DCT (IDCT) is carried out on the frequency regionextended as shown in FIG. 16(D) to be transformed to an image region. Atthis time, the image region has an image size of nmN×nmN, and a portiona of mN×mN pixels at the center is a magnified image.

In step S9, the number of iteration times is updated. In step S10, theregion, indicated by × signs, outside the mN×mN-pixel portion α at thecenter in FIG. 16(E) is corrected to a not-clear but predetermined value“L” by IDCT. Thus, the state of FIG. 16(F) is attained.

This operation is called spatial region limitation. When DCT is carriedout on the image in FIG. 16(F) having the corrected extension region instep S11, the frequency component b shown in FIG. 16(G) can be obtained.

In step S12, a low frequency region of the frequency component bobtained in step S11 is substituted by a known value a to attain thestate of FIG. 16(H).

In step S13, IDCT is carried out on the region including frequencycomponents a and b to obtain the image in FIG. 16(I). In step S14, adetermination is made as to whether the number of iteration timesexceeds a preset value and, when it does not, the processing from stepS9 to step S13 is repeatedly performed.

When the number of iteration times exceeds the value in step S14, themagnified image is output in step S15, and all the operation iscompleted.

In the conventional technique described above, an original image is notdivided but it is transformed at a time. When DCT transform is carriedout on a large sized image, however, enormous processing time isrequired, which makes the conventional method non-practical.

Accordingly, the method of once dividing an original image into smallsized image blocks and then carrying out resolution conversionprocessing in each block has been proposed.

FIG. 17 schematically shows how an original image block is cut out andan extension region is set in the conventional IM-GPDCT processing.

Referring to FIG. 17, an original image (#601) is divided into images ofpredetermined N×N pixels (#602) by block division processing. Here, thecut-out block to be processed is called a target block (#603). Theentire original image is processed by causing all blocks to be targetblocks. A case where a block near the center of the character in theoriginal image is cut out will be described as an example.

An extension region of nN×nN pixels is added to the target block (#604),and the resolution conversion processing thereafter is carried out(#605).

The extension region data is fixed to a particular value as describedabove. In the conventional method, “0,” “255,” or the average value ofimage data in the target block is generally set as the extension regiondata.

First Problem

FIG. 18 shows three-dimensional image data in a target block. Here, areflection factor is adopted as image data, and the target block isformed of eight pixels in both of main and sub scanning directions.

As is apparent from FIG. 18, the reflection factor is higher on the farleft side and lower toward the near right side in the image data in thetarget block.

In the following, problems with the conventional technique will bedescribed based on a case where an image on the cross section (crosssection A indicated by the dashed line in the figure) of the fourthpixel in the sub scanning direction of the target block is to beprocessed.

FIG. 19 shows charts for describing problems with a case where imagedata (reflection factor) in the extension region is set to 255 and theIM-GPDCT processing is carried out in the conventional technique.

Referring to the figure, a) shows relations between a pixel position andits image data on cross section A of the target block in FIG. 18, and b)shows a state where image data L=255 is added to the extension region ofthe original image. In this case, the extension region data has anunnaturally large value as compared with the original image data, and alarge edge is created at a boundary between the original image regionand the extension region.

When resolution conversion is performed on the image consisting of thetarget block and the extension region by the IM-GPDCT method, largewinding is created near the edge as shown in c). In other words, theIM-GPDCT method is intended to restore a high frequency component thatis lost during a sampling operation, and therefore a high frequencycomponent particularly exists at the edge portion and winding is createdbecause of an attempt to restore the high frequency component.

Especially when blocks are cut out and then combined together afterresolution conversion in order to make the processing faster, ringing iscaused in each block. Therefore, a block noise is caused in aconspicuous manner in the conventional technique.

FIG. 20 shows charts for describing problems with a case where imagedata L of the extension region is set to 0 in the conventionaltechnique.

In this case as well, the image data of L=0 is added as the extensionregion as shown in b), and therefore an excessive edge is created at aboundary between the extension region and the original image region andunnatural winding is found when the IM-GPDCT method is adopted.

FIG. 21 shows charts for describing problems with a case where the imagedata of the extension region is set to the average value of image dataincluded in the target block in the conventional technique.

As shown in b), an edge portion is created between the extension regionand the original image region even when the average value of image datain the original image region of the target block is used for theextension region. Further, the IM-GPDCT processing causes unnaturalwinding and an image noise as shown in c).

Second Problem

The IM-GPDCT method also has relatively long processing time. When theBilinear method, for example, which has relatively short processingtime, is used, however, the high frequency component of an image cannotbe restored.

SUMMARY OF THE INVENTION

The present invention was made to solve the problems above and its firstobject is to prevent noise generation in a pixel interpolating device.

A second object of the present invention is to improve processing speedin a pixel interpolating device capable of restoring a high frequencycomponent.

In order to achieve the objects above, according to one aspect of thepresent invention, a pixel interpolating device for interpolating apixel by restoring a lost high frequency component under the tworestrictive conditions that information on a passing frequency band iscorrect and an expanse of an image is limited in a process in whichnormal transformation and inverse transformation of an image arerepeated by orthogonal transformation includes a cutting-out unit forcutting out a target block from an original image, and a setting unitfor setting the data of an extension region of the target block, whichis required for pixel interpolation, based on image data peripheral tothe target block of the original image.

According to the present invention, noise generation can be prevented inthe pixel interpolating device.

According to another aspect of the present invention, a pixelinterpolating device for interpolating a pixel in an input imageincludes a determining unit for determining whether an edge portionexists in the input image, and a switching unit for switching a pixelinterpolation method for the input image based on the determinationresult of the determining unit.

According to the present invention, the pixel interpolation method isswitched based on the determination result as to whether an edge portionexists, and therefore the processing speed of the device can beimproved.

According to still another aspect of the present invention, a pixelinterpolation method includes the steps of a) cutting out a target blockfrom original image data, b) adding an extension region around thetarget block to obtain an extension block, the data of the extensionregion being set based on image data peripheral to the target block ofthe original image, c) carrying out DCT transform on the image data ofthe extension region to obtain a frequency component, d) extending theobtained frequency component to a high frequency region and setting theinitial value of the frequency component of the high frequency region to0, e) carrying out inverse DCT transform on the frequency componentobtained by extending the frequency component in d) to obtain the imagedata of a magnified extension block, the magnified extension blockincluding a magnified target block at the center of the magnifiedextension block, f) setting data based on the image data peripheral tothe target block of the original image for a peripheral region of themagnified target block in the magnified extension block, g) carrying outDCT transform on the image data of the extension block obtained in f) toobtain a frequency component, h) substituting a low frequency region ofthe frequency component obtained in g) by the frequency componentobtained in c), and i) carrying out inverse DCT transform on thefrequency component obtained in h) to obtain image data, magnified imagedata being obtained by repeating the steps from f) to i) a prescribednumber of times.

According to still another aspect of the present invention, a pixelinterpolating method includes the steps of determining whether an edgeportion exists in an input image, selecting one of the first and secondpixel interpolation methods based on the determination result, andcarrying out pixel interpolation of the input image by the selectedpixel interpolation method.

According to still another aspect of the present invention, an imageprocessing apparatus for interpolating a pixel by restoring a highfrequency component by repeating normal transformation and inversetransformation of an image through orthogonal transformation includes acutting-out unit for cutting out a target block from original imagedata, a setting unit for setting an extension block by adding anextension region around the target block, the data of the extensionregion being determined based on image data peripheral to the targetblock of the original image, a transforming unit for carrying out DCTtransform on the image data of the extension block to obtain a frequencycomponent, a frequency extending unit for extending the obtainedfrequency component to a high frequency region and setting a prescribedvalue as the initial value of the frequency component of the highfrequency region, and an inverse transforming unit for carrying outinverse DCT on the frequency component obtained by extending thefrequency region to obtain magnified image data.

According to still another aspect of the present invention, a pixelinterpolating device includes a first pixel interpolating unit forinterpolating a pixel by restoring a high frequency component of animage, which is lost during sampling, a second pixel interpolating unitdifferent from the first pixel interpolating unit, a determining unitfor determining whether an edge portion exists in an input image, and aselecting unit for selecting one of the first and second pixelinterpolating units based on the determination result.

The foregoing and other objects, features, aspects and advantages of thepresent invention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a pixel interpolating device in a firstembodiment of the present invention.

FIG. 2 illustrates processing in the first embodiment.

FIG. 3 is a flow chart of resolution conversion processing in the firstembodiment.

FIG. 4 illustrates effects in the first embodiment.

FIG. 5 is a diagram for describing the processing of an image processingapparatus in a second embodiment of the present invention.

FIG. 6 is a flow chart showing the processing of setting an extensionregion in the second embodiment.

FIG. 7 illustrates effects in the second embodiment.

FIG. 8 is a diagram for describing the processing of an image processingapparatus in a third embodiment of the present invention.

FIG. 9 is a flow chart showing the processing of setting an extensionregion in the third embodiment.

FIG. 10 illustrates the effects of the image processing apparatus in thethird embodiment.

FIG. 11 is a block diagram showing a structure of a resolutionconverting portion 403 in a fourth embodiment of the present invention.

FIG. 12 is a flow chart of resolution conversion processing in thefourth embodiment.

FIG. 13 illustrates a resolution conversion method not reproducing ahigh frequency component.

FIG. 14 illustrates the principle of the IM-GPDCT method.

FIG. 15 is a flow chart showing conventional resolution conversionprocessing.

FIG. 16 schematically shows the IM-GPDCT method.

FIG. 17 illustrates the conventional processing of the IM-GPDCT method.

FIG. 18 shows a specific example of image data of a target block, whichis an object for image processing.

FIG. 19 illustrates problems when data I of an extension region is 255.

FIG. 20 illustrates problems when data I of an extension region is 0.

FIG. 21 illustrates problems when data I of an extension region is theaverage value of pixel data included in a target block.

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

FIG. 1 is a block diagram of an image processing apparatus in a firstembodiment of the present invention. The image processing apparatuscarries out magnification of an image by interpolating pixels.

Referring to the figure, the image processing apparatus includes aninput interface portion 401 for receiving image data to be processed, aninput image memory portion 402 for storing the input image data once, aresolution converting portion 403 for converting resolution, a buffermemory 404, an output image memory portion 405 for storing an outputimage once, an output interface portion 406 for outputting image data,and a CPU 407 for controlling the entire image processing apparatus.

Image data that is input from an external unit through input interfaceportion 401 is once stored in input image memory portion 402. The storedimage data is read as necessary according to the process, and processedin resolution converting portion 403. Since the image is buffered duringthe converting processing at this time, buffer memory 404 stores part ofthe image data. After all the processing is completed, the image thathas gone through prescribed resolution conversion is stored in outputimage memory portion 405 and output through output interface portion 406to a processing portion, such as a printing device, at a subsequentstage.

FIG. 2 illustrates how an original image block is cut out and anextension region is set in the resolution converting portion 403 of theimage processing apparatus in this embodiment.

Original image data (#701) input through input interface portion 401 isdivided into blocks consisting of predetermined N×N pixels in resolutionconverting portion 403 (#702). The blocks are successively cut out inthe block cutting-out portion 4030 and made an object of imageprocessing. A block that is an object of image processing is called atarget block (#703). It is assumed in FIG. 2 that the original image,(#701) is a character image and a block near the center of the characterit is cut out as a target block.

In this embodiment, an image of nN×nN pixels (peripheral image)peripheral to the target block of the original image is separately cutout differently from conventional examples, and the image is made thedata of an extension region (#704) in the setting unit 4035. The imagedata of the target block and the image data of the extension regionconstitute an image of nN×nN pixels (#705), and resolution conversionprocessing by IM-GPDCT is carried out (#707).

After DCT transform and IDCT transform, substitution of the extensionregion is carried out (S10 in FIG. 15). At this time, the data ofmagnified peripheral image (#704) is used as the extension region data(#706). Magnification processing at this time may be carried out by atypical known magnification technique such as the Nearest Neighbormethod and the Bilinear method. These methods are simple pixelinterpolation methods that do not reproduce a high frequency componentof an image in magnification processing.

FIG. 3 is a flow chart of resolution conversion processing performed bythe image processing apparatus in this embodiment.

Referring to the figure, the number of iteration times in the IM-GPDCTmethod and the resolution conversion rate (magnification rate) are setin step S501. In step S502, an original image is then read from inputimage memory portion 402. In step S503, the process of cutting out ablock, for dividing the original image into pieces of a prescribed size,is carried out.

In step S504, a peripheral image (#704 in FIG. 2) of the target block,which has been cut out, is cut out to a prescribed size to be used as anextension region. In step S505, the cut-out peripheral image is added asthe extension region of the target block and, in step S506, DCTtransform is carried out on the target block and the image data of theextension region.

Although the target block and its peripheral image are separately cutout in this embodiment, an image of a prescribed size including thetarget block and its peripheral image may be cut out in advance from anoriginal image and DCT transform may be carried out thereafter.

After DCT transform processing, a known frequency component is oncestored in buffer memory 404 in step S507. Further, the known frequencycomponent is further extended to a high frequency component in stepS508, and “0” is substituted in the high frequency region.

In step S509, inverse DCT transform (IDCT) of data having the extendedhigh frequency region is carried out. Thus, an image formed byconverting the resolution of the original image can be obtained. Inother words, since the number of image data (number of pixels) areincreased by resolution conversion, the image size can be made largerwhen the image is output at the same output resolution, andmagnification processing is made possible. When the size of the outputimage is made the same as that of the original image, the number ofimage data is increased. Thus, resolution can be improved.

Since the extension region peripheral to the IDCT-transformed image hasa known value, substitution is carried out. Although the image data inthe extension region has only a prescribed value in the conventionaltechnique, an image peripheral to the target block is used forsubstitution in this embodiment. Since the image peripheral to thetarget block and the resolution-converted image have different sizes,they cannot be substituted as they are. Accordingly, the imageperipheral to the target block is magnified in advance in step S510. Formagnification, an interpolation method that does not reproduce a highfrequency component as described above is used.

In step S511, the number of iteration times is incremented by 1. In stepS512, the image magnified in step S510 is written to the extensionregion. In step S513, DCT transform is carried out.

Since a low frequency region of the data transformed to a frequencycomponent is the known data stored in buffer memory 404 in step S507,substitution of data in a low frequency band is carried out in stepS514, and inverse DCT transform is carried out again in step S515.

In step S516, a determination is made as to whether the current numberof iteration times is the number of iteration times that is set in stepS501, and the processing from step S511 is repeated till the iterationtime number attains the set number. When the iteration time numberattains the set number, the image of the transformed target block iswritten to output image memory portion 405, and processing of the nexttarget block is started.

When processing of all the blocks is completed, conversion processingends.

FIG. 4 illustrates the effects of the image processing apparatus in thisembodiment, and corresponds to FIGS. 16 to 18.

In FIG. 4, a) shows a state where the image data of a target block is ofcross section A. In this embodiment, an image peripheral to the targetblock is added as an extension region to the region of an original imageshown in a), as shown in b). Accordingly, an unnatural edge is notcreated between the extension region and the original image region. Thatis, the image has natural continuity even after the extension region isadded.

As shown in c), an image having the image peripheral to the originalimage as the extension region does not have image winding, caused by anedge, at a boundary between the extension region and the original imageregion differently from the conventional technique, even when a highfrequency component is restored. Accordingly, an edge noise for eachblock image is not caused even after combination of blocks, andtherefore a block noise that may occur in the entire image can beprevented.

When an original image actually includes a high frequency component, thehigh frequency component is restored by the IM-GPDCT method.Accordingly, a blurred image can be prevented even when the image ismagnified or resolution is converted.

Since image processing is performed after blocks are cut out from anoriginal image in this embodiment, it is not necessary to carry out DCTtransform on a large sized original image at a time. Thus, timenecessary for image processing can be shortened.

Second Embodiment

The block diagram of an image processing apparatus in a secondembodiment of the present invention is similar to that of the firstembodiment shown in FIG. 1.

FIG. 5 is a diagram for describing processing carried out by the imageprocessing apparatus in the second embodiment.

Referring to FIG. 5, a target block (image target block) is cut out inblock cut-out processing, and then an extension region necessary for DCTtransform is added. The second embodiment is characterized in that imagedata peripheral to the target block of an original image (image data ofan adjacent block) and data having the degree of change derived from theimage data of the target block are used as the data of the extensionregion. In short, the data of extension region pixels (7), (8) and (9)are calculated from the data of pixel (1) in the target block and thedata of pixel (3) in an adjacent pixel in FIG. 5. Here, such data thatcause a linear change from the data of pixel (1) to the data of pixel(3) are set as the data of pixels (7), (8) and (9). Similarly, the dataof pixels (4) to (6) are calculated from the data of pixels (1) and (2).

The data of pixels in the extension region surrounding-the target blockare calculated in a similar manner. Here, the data of pixels existing inthe range of pixels (10) to (12) in FIG. 5 cannot be found from the dataof adjacent blocks. Accordingly, as the data of pixels in this portionof the extension region, the average value of the adjacent pixel data isused. More specifically, the average value of the data of pixels (4) and(7) is used as the data of pixel (10).

Further, the average value of the data of pixels (5) and (8) is the dataof three pixels in the range of (11). The average value of the data ofpixels (6) and (9) is the data of five pixels in the range of (12).

FIG. 6 is a flow chart showing the processing of setting an extensionregion carried out by the image processing apparatus in the secondembodiment. This flow chart corresponds to steps S504 and S505 of theflow chart in FIG. 3.

Referring to the figure, the values of variables i and n that indicatethe position of an extension region to be processed is set to 1 in stepS20. In step S21, a target block is cut out. In step S22, one of thepixels that are outermost in the target block is picked up as a pixel ofinterest.

In step S23, pixel data in a block adjacent to the extension region isextracted corresponding to the pixel of interest. In other words, pixel(2) or (3) is extracted when the pixel that is to be processed in FIG. 5is pixel (1), for example.

In step S24, the degree of change α is found. The expression (11) belowis used to find the degree of change α.

 α=(a _(n) −A _(n))/(extension region width+1)  (11)

In the expression (11), a_(n) is the data of an adjacent pixel and A_(n)is the data of a pixel of interest. Further, the extension region widthis the width of an extension region that exists between a target blockand an adjacent block, and the width is that of three pixels indicatedby (7) to (9) in FIG. 5.

In step S25, the data of a pixel in the extension region that is closestto the target block is calculated by the expression (12).

β(i)=A _(n)+α  (12)

The expression is intended to find the data of pixel (7) when the dataof the extension region is calculated based on pixels (1) and (3) inFIG. 5.

Then, the value of i is incremented by 1 in step S26. In step S27, thedata of the next pixel in the extension region is calculated by theexpression (13).

β(i)=β(i−1)+α  (13)

This is intended to find the data of pixel (8) when the data of theextension region is found from the data of pixels (1) and (3) in theexample of FIG. 5.

In step S28, a determination is made as to whether the value of iattains the extension region width, and the processing from step S26 isrepeatedly carried out till the value attains it.

When the value of i attains the extension region width in step S28, thepixel of interest is changed by incrementing the value of n by 1 in stepS29. In step S30, the data of β(1) to β(i) already found in step S30 areassigned as the data of the extension region.

In step S31, a determination is made as to whether the entire extensionregion is filled, and the processing from step S22 is repeatedly carriedout till the region is filled.

After the data of the extension region is all found, the IM-GPDCTprocessing is carried out similarly to steps S506 to S517 in FIG. 3.Thus, pixel interpolation is completed.

FIG. 7 illustrates the effects of the image processing apparatus in thisembodiment.

It is assumed that image data on cross section A in FIG. 15 is processedas shown in FIG. 7(a). At this time, the data having the degree ofchange derived from the image data of an adjacent block as shown in b)is set as an extension region in this embodiment.

Accordingly, an unnatural edge as shown in FIGS. 16 to 18 is not createdbetween a target block and the extension region. Thus, after TheIM-GPDCT processing, winding is not caused between the extension regionand the target block (original image) as shown in c). Even when theoriginal image is divided into a plurality of blocks, generation of anedge noise as well as a block noise that spreads to the entire image canbe prevented. Further, the processing time in DCT transform even of alarge sized original image can be shortened by dividing the originalimage into a plurality of blocks.

Third Embodiment

Since the block diagram of an image processing apparatus in a thirdembodiment of the present invention is similar to that of the firstembodiment, the description will not be repeated. The third embodimentis characterized in that an average value of the image data of a targetblock and the image data of an adjacent block is used as pixel data inan extension region that is set around the target block.

More specifically, adjacent blocks existing around the block A ofinterest are a1 to a8 in FIG. 8. When the average value of the data ofpixels included in block A of interest is A(ave), and the average valuesof the data of pixels included in each of adjacent blocks a1 to a8 area1(ave) to a8(ave), respectively, the data of pixels in extensionregions α1 to α8 are set by α1 to α8 as in the expression (14).

α1=[A(ave)+a 1(ave)]/2

α2=[A(ave)+a 2(ave)]/2

α8=[A(ave)+a 8(ave)]/2  (14)

FIG. 9 is a flow chart showing the processing of setting an extensionregion carried out by the image processing apparatus in this embodiment.The flow chart corresponds to steps S504 and S505 in FIG. 3.

Referring to FIG. 9, a target block is cut out in step S121. Then, dataA1 to An of all pixels included in the target block are extracted instep S122.

In step S123, the data of all pixels included in each of adjacent blocksa1 to a8 are extracted. In step S124, an average of the data of pixelsincluded in the target block is calculated.

In step S125, an average of the data of pixels included in each of theadjacent blocks is calculated.

In step S126, each data of extension regions α1 to α8 is calculatedbased on the expression (14).

In step S127, a determination is made as to whether the entire extensionregion is filled by the data and, when it is not, an adjacent block tobe processed is changed in step S128 and the processing from step S123is repeatedly carried out.

When the extension region is filled in step S127, the processing here isfinished. The processing from step S506 in FIG. 3 is performed.

FIG. 10 illustrates the effects of the image processing apparatus inthis embodiment.

Referring to FIG. 10a), it is assumed that the data of an original imageis of cross section A in FIG. 15. At this time, an average value of theaverage value of the data of pixels included in an adjacent block andthe average value of the data of pixels included in a target block isused as the data of an extension region as shown in b) in thisembodiment. Thus, an unnatural edge is not created between the targetblock and the extension region as shown in FIG. 10c), and winding isalso not caused even after the IM-GPDCT processing. Accordingly, similareffects to those of the first and second embodiments can also beattained in this embodiment.

Fourth Embodiment

Since the overall structure of an image processing apparatus in a fourthembodiment is the same as the one shown in FIG. 1, the description willnot be repeated.

In this embodiment, the structure shown in FIG. 11 is adopted asresolution converting portion 403.

FIG. 11 is a block diagram showing a specific structure of resolutionconverting portion 403 in FIG. 1.

Resolution converting portion 403 includes a block cutting-out portion4030 for cutting out a block from image data stored in input imagememory portion 402, an edge determining portion 4031 for determiningwhether an edge component is included in the cut-out block, an IM-GPDCTportion 4032 for carrying out pixel interpolation on the image of thecut-out block using the IM-GPDCT method, a CC portion 4033 for carryingout pixel interpolation on the image of the cut-out block by the CubicConvolution method (hereinafter, referred to as the CC method), and aselector 4034 for selecting a preferred one of the output of IM-GPDCTportion 4032 and the output of CC portion 4033.

When an edge component exists in the image of a cut-out block, the imageprocessing apparatus in this embodiment adopts the IM-GPDCT methodrestoring a high frequency component, which is lost during a samplingoperation, under the two restrictive conditions that information on apassing frequency band is correct and an expanse of an image is limited.When an edge component does not exist, the image processing apparatusadopts the CC method that is an interpolation method not restoring ahigh frequency component of image data. Accordingly, a lost highfrequency component can be restored and high speed image processing canbe made possible in the image processing apparatus in this embodiment.

Although the Cubic Convolution method is adopted as an interpolationmethod not restoring a high frequency component in this embodiment, aprocessing portion that adopts the Nearest Neighbor method or theBilinear method, as other representative resolution conversion methods,may be employed in stead of CC portion 4033.

FIG. 12 is a flow chart showing the image magnification processingcarried out by the image processing apparatus in this embodiment.

Referring to the figure, the number of iteration times in the IM-GPDCTmethod and the magnification rate are set in step S101. In step S102, aninput image is read through input interface portion 402 to input imagememory portion 402. In step S103, a block is cut out from the image datastored in input image memory portion 402 by block cutting-out portion4030.

In step S104, a determination is made as to whether an edge portion(edge component) exists in the image of the cutout block. When it does,the same processing (image processing using the IM-GPDCT method) assteps S4 to S15 in FIG. 15 is carried out in steps S105 to S116.

When an edge portion does not exist, resolution is converted by a method(such as the CC method) that does not restore a high frequency componentin step S117. Thereafter, the processed image is stored in output imagememory portion 405 through selector 4034 in step S116.

FIG. 13 illustrates the resolution conversion method, which does notrestore a high frequency component, carried out in step S117 in FIG. 12.In the figure, (A) shows processing by the Nearest Neighbor method, (B)shows processing by the Bilinear method, and (C) shows the CubicConvolution processing. In each of them, the value of image data (pixeldensity) f(x) is determined at pixel position x=−1, 0, 1. It is assumedthat pixel data is interpolated between pixels (x=−0.5, 0.5, forexample).

Referring to (A), in the Nearest Neighbor method, the value of a pixelclosest to a pixel to be interpolated (point of interest) is adopted asit is as the image data of the pixel to be interpolated.

Referring to (B), in the Bilinear method, the value of a peripheralpixel is linearly changed according to the distance from a pixel to beinterpolated, and a value on the line is adopted as the image data ofthe pixel to be interpolated.

Referring to (C), in Cubic Convolution, the degree to which the imagedata of an peripheral pixel is reflected is changed in a curve manneraccording to the distance from a pixel to be interpolated, and data onthe curve is used.

In the embodiment, a block is cut out and, based on whether an edgecomponent exists in the cut-out block, an image processing method isswitched. However, the processing method for the entire image may beswitched according to the determination result as to whether an edgecomponent exists in any portion of an input image data, without cuttingout a block.

Although the present invention has been described and illustrated indetail, it is clearly understood that the same is by way of illustrationand example only and is not to be taken by way of limitation, the spiritand scope of the present invention being limited only by the terms ofthe appended claims.

What is claimed is:
 1. A pixel interpolating device for interpolating apixel by restoring a lost high frequency component under two restrictiveconditions that information on a passing frequency band is correct andan expanse of an image is limited in a process in which normaltransformation and inverse transformation of the image is repeated byorthogonal transformation, the pixel interpolating device comprising: acutting-out unit for cutting out a target block from a target region ofan original image; and a setting unit for setting data of an extensionregion, which is required for pixel interpolation, of the thus cut outtarget block, wherein the thus set extension region data is based onimage data that is peripheral to the target region of said originalimage.
 2. A pixel interpolating device according to claim 1, whereinsaid setting unit uses the image data peripheral to the target region ofsaid original image as the data of the extension region of said targetblock.
 3. A pixel interpolating device according to claim 2, furthercomprising a magnifying unit for magnifying the image data peripheral tothe target region of said original image by an interpolation method notrestoring a high frequency component, wherein said setting unit uses theimage data magnified by said magnifying unit.
 4. A pixel interpolatingdevice according to claim 1, wherein said setting unit uses data havinga degree of change derived from the image data peripheral to the targetregion of said original image and image data of the target block as thedata of the extension region of said target block.
 5. A pixelinterpolating device according to claim 1, wherein said setting unituses an average value of the image data of the target block of saidoriginal image and image data peripheral to the target region of saidoriginal image as the data of the extension region of said target block.6. A pixel interpolating device according to claim 1, wherein saidorthogonal transformation is discrete cosine transform (DCT).
 7. A pixelinterpolating device adapted for carrying out a plurality of pixelinterpolation methods in an input image, comprising: a determining unitfor determining whether an edge portion exists in said input image; anda selecting unit for selecting one of the plurality of pixelinterpolation methods for said input image based on the determinationresult of said determining unit, wherein said plurality of pixelinterpolation methods includes a first pixel interpolation methodincluding the steps of: cutting out a target block from a target regionof original image data, adding an extension region around the targetblock to obtain an extension block, data of the extension region beingset based on image data peripheral to the target region of the originalimage data.
 8. A pixel interpolating device according to claim 7,wherein said selecting unit selects the first pixel interpolation methodwhen an edge portion exists in said input image and selects a secondpixel interpolation method when an edge portion does not exist in saidinput image, wherein said first pixel interpolation method restores ahigh frequency component, which is lost during sampling, under tworestrictive conditions that information on a passing frequency band iscorrect and an expanse of an image is limited in a process in whichnormal transformation and inverse transformation of the image isrepeated by orthogonal transformation, and wherein said second pixelinterpolation method is an interpolation method not restoring a highfrequency component of image data.
 9. A pixel interpolating deviceaccording to claim 7, further comprising a block cutting-out unit forperforming the step of cutting out the target block from said inputimage, wherein said determining unit determines whether an edge portionexists in the thus cut-out block.
 10. The pixel interpolation deviceaccording to claim 8, wherein said orthogonal transformation is discretecosine transform (DCT).
 11. A pixel interpolation method, comprising thesteps of: a) cutting out a target block from a target region of originalimage data; b) adding an extension region around the target block toobtain an extension block, data of the extension region being set basedon image data peripheral to the target region of the original image; c)carrying out a discrete cosine transform on image data of said-extensionblock to obtain a frequency component; d) extending the thus obtainedfrequency component to a high frequency region to obtain a broadfrequency component, and setting an initial value of a frequencycomponent of the high frequency region to 0; e) carrying out an inversediscrete cosine transform on the broad frequency component obtained byextending a frequency region in d) to obtain image data of a magnifiedextension block, the magnified extension block including a magnifiedtarget block at a center of the magnified extension block; f) settingdata of a peripheral region of the magnified target block in themagnified extension block based on the image data peripheral to thetarget region of the original image; g) carrying out a discrete cosinetransform on image data of the extension block obtained in f) to obtaina frequency component; h) substituting a low frequency region of thefrequency component obtained in g) with the frequency component obtainedin c); and i) carrying out an inverse discrete cosine transform on thefrequency component obtained in h) to obtain image data, whereinmagnified image data is obtained by repeating said steps f) to i) aprescribed number of times.
 12. A pixel interpolation method accordingto claim 11, wherein the image data peripheral to the target region ofthe original image is used as the data of the extension region in thestep b).
 13. A pixel interpolation method according to claim 12, whereindata formed by converting a pixel density of the image data peripheralto the target region of said original image is used as the data of theperipheral region in the step f).
 14. A pixel interpolation methodaccording to claim 11, wherein data having a degree of change calculatedfrom the image data peripheral to the target region of said originalimage and image data of the target block of said original image is usedas the data of the peripheral region in the step b).
 15. A pixelinterpolation method according to claim 11, wherein an average value ofthe image data of the target block of said original image and the imagedata peripheral to the target region of said original image is used asthe data of the peripheral region in the step b).
 16. A pixelinterpolation method, comprising the steps of: determining whether anedge portion exists in an input image, selecting one of a first pixelinterpolation method and a second pixel interpolation method based onthe result obtained in the determining step; and carrying out pixelinterpolation on the input image by the selected pixel interpolationmethod, wherein said first pixel interpolation method includes the stepsof: a) cutting out a target block from a target region of original imagedata, b) adding an extension region around the target block to obtain anextension block, data of the extension region being set based on imagedata peripheral to the target region of the original image data.
 17. Apixel interpolation method according to claim 16, wherein said firstpixel interpolation method further includes the steps of: c) carryingout a discrete cosine transform on image data of said extension block toobtain a frequency component, d) extending said obtained frequencycomponent to a high frequency region and setting an initial value of afrequency component of the high frequency region to 0, e) carrying outan inverse discrete cosine transform on the frequency component obtainedby extending the frequency region in d) to obtain image data of amagnified extension block, the magnified extension block including amagnified target block at a center of the magnified extension block, f)setting data of a peripheral region of the magnified target block in themagnified extension block based on the image data peripheral to thetarget region of the original image, g) carrying out a discrete cosinetransform on image data of the extension block obtained in f) to obtaina frequency component, h) substituting a low frequency region of thefrequency component obtained in g) by the frequency component obtainedin c), and i) carrying out an inverse discrete cosine transform on thefrequency component obtained in h) to obtain image data, whereinmagnified image data is obtained by repeating said steps f) to i) aprescribed number of times.
 18. An image processing apparatus forcarrying out pixel interpolation by restoring a high frequency componentby repeating normal transformation and inverse transformation of animage through orthogonal transformation, comprising: a cutting-out unitfor cutting out a target block from a target region of original imagedata; a setting unit for setting an extension block by adding anextension region around the target block, data of the extension regionbeing determined based on image data peripheral to the target region ofthe original image; a transforming unit for performing a discrete cosinetransform on image data of said extension block to obtain a frequencycomponent; a frequency extending unit-for extending the thus obtainedfrequency component to a high frequency region and setting a prescribedvalue as an initial value of a frequency component of the high frequencyregion; and an inverse transforming unit for performing an inversediscrete cosine transform on the thus extended frequency component toobtain magnified image data.
 19. An image processing apparatus accordingto claim 18, wherein said setting unit sets the image data peripheral tosaid target region of said original image as the data of said extensionregion.
 20. An image processing apparatus according to claim 18, whereinsaid setting unit sets data having a degree of change calculated fromthe image data peripheral to the target region of said original imageand image data of the target block of said original image as the data ofsaid extension region.
 21. An image processing apparatus according toclaim 18, wherein said setting unit sets an average value of image dataof the target block of said original image and the image data peripheralto the target region of said original image as the data of saidextension region.
 22. An image processing apparatus, comprising: a firstpixel interpolating unit for carrying out pixel interpolation byrestoring a high frequency component of an image which is lost duringsampling; a second pixel interpolating unit for carrying out a pixelinterpolation method different from the first pixel interpolating unit;a determining unit for determining whether an edge portion exists in aninput image; and a selecting unit for selecting one of said first andsecond pixel interpolating units based on the thus obtaineddetermination result, wherein said first pixel interpolating unitincludes: a cutting-out unit for cutting out a target block from atarget region of original image data; a setting unit for setting anextension block by adding an extension region around the target block,data of the extension region being determined based on image dataperipheral to the target region of the original image.
 23. An imageprocessing apparatus according to claim 22, wherein said first pixelinterpolating unit further includes: a transforming unit for performinga discrete cosine transform on image data of said extension block toobtain a frequency component; a frequency extending unit for extendingthe thus obtained frequency component to a high frequency region andsetting a prescribed value as an initial value of a frequency componentof the high frequency region; and an inverse transforming unit forperforming an inverse discrete cosine transform on the thus extendedfrequency component to obtain magnified image data.
 24. A pixelinterpolating device according to claim 1, wherein the target block issubstantially the same size as the target region.
 25. A pixelinterpolation method according to claim 11, wherein the target block issubstantially the same size as the target region.
 26. An imageprocessing apparatus according to claim 18, wherein the target block issubstantially the same size as the target region.
 27. An imageprocessing apparatus according to claim 23, wherein the target block issubstantially the same size as the target region.